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Running  head.  Functions  with  the  same  optimal  knots. 


Abstract.  The  relationship  between  functions  with  the  same  optimal  knots 
for  L2{0,1]  approximation  by  k-th  order  splines  or  piecewise  polynomials 
is  investigated.  It  is  shown  that,  if  two  functions  have  positive  con¬ 
tinuous  k-th  derivatives,  they  will  have  the  same  optimal  knots  if  and 
only  if  they  differ  by  a  polynomial  of  order  k.  An  application  to  design 
selection  for  continuous  time  regression  is  considered  and  extensions  to 
L  approximation  are  also  provided. 


1.  Introduction  Consider  the  LjlO.l]  approximation  of  two  functions  by 
splines  or  piecewise  polynomials  with  free  knots.  In  this  note  we  investigate 
the  relationship  between  functions,  f  and  g,  having  the  same  optimal 


knots. 

Ic  lc 

Let  P„  and  S„  denote,  respectively,  the  set  of  all  k-th  order  piecewise 
N  N 

polynomials  and  k-th  order  splines  with  N-l  distinct  knots  in  the  interior 

of  [0,11.  For  simplicity,  a  generic  notation,  f„,  will  be  used  to  indicate 

k  k 

the  best  L2[0,l]  approximation  to  f  from  either  PN  or  the  closure  of  S^, 

Sjjj.  its  meaning  will  be  clear  from  context.  The  properties  of  f^  for  PN 
have  been  studied  by  Chow  [5]  and  Burchard  and  Hale  [4].  Similar 
results  for  can  be  found  in  Barrow,  et  al.  [2]  and  Barrow  and  Smith  [3]. 
All  these  papers,  however,  have  focused  on  the  behavior  of  f^  and  its 
corresponding  optimal  knots  for  f  in  various  function  classes.  For  instance, 
Chow  [5]  has  established  conditions  on  f  that  insure  unique  optimal  knots 
for  each  N  whereas  Barrow  and  Smith  [3]  have  provided  an  asymptotic 
characterization  of  the  behavior  of  the  optimal  knots  in  terms  of  f's 
k-th  derivative.  In  constrast,  this  paper  focuses  on  what  is,  in  a  certain 
sense,  an  inverse  of  these  problems.  Rather  than  asking  how  f's  properties 
characterize  the  behavior  of  it's  optimal  knots  we  ask,  instead,  to  what 
extent  the  optimal  knots  characterize  f.  The  answer  is,  essentially,  that 
two  functions  with  the  same  optimal  knots  for  all  N  must  differ  by  at  most 
a  k-th  order  polynomial.  More  specifically  we  show  the  following: 

Theorem  1.  Let  f  ,g  6  L-[0,1]  nck[0,l]  with  f  ^  and  both  positive  on 


[0,1].  Then,  f and  g  have  the  same  optimal  knots  for  L2[0,l]  approximation 
k  -k 

from  Pjj  (or  from  Sjj)  for  all  N  _if  and  only  if  there  exists  constants  Oq,  , . 


(o^O)  such  that 

f(t)  *  +  “k*^  ’ 


The  proof  of  Theorem  1  is  given  in  the  next  section.  Although  this 


2 


theorem  is  of  interest  in  its  own  right  it  also  has  statistical  applications, 

one*of  which  is  briefly  discussed  in  Section  3.  Another  application  is  to  the 

statistical  problems  of  grouping,  spacing  and  stratification  considered  by 

Adatla  and  Chan  [1]  who,  when  k«l,  have  obtained  a  special  case  of  this 

result.  It  follows  from  Eubank  [6]  that  Theorem  1  can  be  used  to  obtain 

their  Theorem  5  under  alternative  conditions.  Section  4  contains  some  concluding 

remarks  and  suggestions  for  future  research.  An  extension  of  Theorem  1  to 

L  [0,1]  approximation  is  also  provided. 

P 

2.  Proof  of  Theorem  Let  us  first  prove  Theorem  1  for  approximation  from 
-k  k 

SN  as  t*ie  case  will  follow  similarly.  Throughout  this  and  the  sub¬ 

sequent  section  we  use  ||*||  to  denote  the  usual  L»[0,1]  norm  and  D„  for  the 
set  of  'all  possible'  knot  choices,  l.e., 

Djj  m  (  (^0*  t^»  •  •  • ,  tjj)  :  <  tj  <•••<  (2.1) 

Jr 

For  TeDN  let  S,j,  denote  the  L2[0,l]  orthogonal  projector  for  the  set  of  all 
k-th  order  splines  having  knots  at  T.  Then,  a  best  approximation  fN  is 
characterized  by 

I  |f-fNl  I  *  iBfT*D  llWTfH*  (2’2) 

Note  that  although  S,j,f  has  noncoincident  knots,  the  knots  for  fN  need  not 
be  distinct. 

Now  let  f(t)  •  <*jt*  +  *or  some  constants  °q . “jj  with 

Then  ||f-S^fj|  -  |ojJ  ||g-S^g||  for  all  T e D^.  Consequently,  f 
and  g  have  the  same  optimal  knots  for  all  N  and  the  direct  implication  has 
been  verified. 

To  see  the  necessity  of  condition  (1.1)  let  {fN>  be  a  sequence  of  best 
approximations  to  f  and  0  -  tQ  <  ^  <•••<  t”_1  <  t”  «  1  the  optimal  knots 
corresponding  to  f^.  By  assumption  these  will  also  be  optimal  knots  for  g. 
Define,  as  in  Barrow  and  Smith  [3],  a  piecewise  linear  function  TNe[0,l] 

satisfying  T^(i/N)  •  t^,  i  ■(),..., N,  and  let  s^  denote  Its  right  continuous 

(k) 

Inverse.  It  then  follows  from  Theorem  3  of  [3]  that,  since  f  >  0,  as 
N  ■*  • 
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VT) 


/  {f(k)<t)>2/2k+1dt//  {f(k)<x)}2/2k+1dx,  uniformly  on  [0,1] 
0  0 


(2.3) 


N  N 

But,  since  the  {t^}j_Q  ace  also  optimal  knots  for  g, 

Sj.(t)  -*>  /  {g(k)(t)}2/2k+1dt//1{g(k)(x)}2/2k+1dj 
0  0 


(2.4) 


The  result  now  follows  by  equating  (2.3)  and  (2.4)  and  differentiating. 

To  prove  Theorem  1  for  we  need  only  note  that  an  analog  of  Theorem  3 
In  Barrow  and  Smith  [3]  can  be  shown  to  hold,  in  this  case,  by  using  their 
proof  in  conjunction  with  Theorem  1.1  of  [4].  The  required  result  can 
then  be  obtained  as  in  the  previous  arguments. 

3.  An  application  to  regression  design  for  time  series.  In  this  section 
an  application  of  Theorem  1  to  the  problem  of  optimal  design  for  continuous 
time  regression  will  be  presented.  The  situation  is  as  follows.  A 
stochastic  process,  Y,  is  assumed  to  have  the  form 


Y (t)  -  Bf(t)  +  X(t) V te[0,l] , 


(3.1) 


where  f  is  a  known  regression  function,  B  is  an  unknown  parameter  and  X 


is  a  zero  mean  process  with  covariance  kernel 
R(s, t)  -  (k-1)!'2/  (s-u)k~1(t-u)k_1du, 

n  ^  + 


(3.2) 


k-i  k-1 

with  x+  »x  if  x  >  0  and  zero  otherwise.  The  objective  is  to  obtain  an 

estimate  of  the  unknown  parameter  S.  However,  suppose  it  is  possible  only 

to  sample  the  Y  process,  as  well  as  it  k-1  quadratic  mean  derivatives,  at 

some  set  of  noncoincident  time  points  TcD^.  Thus  an  estimator  must  be 

k  (i) 

based  solely  on  the  observations  Y^,  ■  (Y  (t):teT,  i-0, . . . ,k-l}.  For  this 
purpose  the  best  linear  unbiased  estimator  of  6  can  be  obtained  through 
the  use  of  generalized  least  squares  (c.f.  Sacks  and  Ylvlsaker  J10].) 

A 

This  estimator  will  be  denoted  by  6.  _.  Since  the  choice  of  T  is  usually 

T 

at  the  discretion  of  the  experimenter,  the  problem  arises  as  to  how  T  should 

A  A 

be  selected.  Let  7(8^  j)  denote  the  variance  of  8^  then, an  optimal 
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estimator  can  be  obtained  by  choosing  T*eD^  to  satisfy 

V(SkiI.)  -  i»f  ) .  (3.3) 

N 

Under  the  regularity  condition  that  f  admits  the  representation 

f(t)  -  (H)l‘1/1f(k)(u)(t-u)^1du  (3.4) 

0  + 

(k) 

for  some  f  eL2[0,1]  problem  (3.3)  is  amenable  to  analysis  and  has  been 

studied  by  Sacks  and  Ylvisaker  [10],  Wahba  [11],  Eubank,  Smith  and  Smith  [7] 

and  others.  We  now  investigate,  with  the  aid  of  Theorem  1,  the  relationship 

between  regression  models  having  identical  optimal  designs  for  all  N. 

It  fqllows  from  Eubank,  Smith  and  Smith  [7]  that  for  R  of  the  form 

(3.2),  TeDN  and  f  satisfying  (3.4)  we  may  write  V(8^  T)  «| |P^f(k) | |  ^  where 

is  the  I^tO.l]  orthogonal  projector  for  the  set  of  all  piecewise 

polynomials  of  order  kwithknots  at  T.  As  |  |H£f^  |  |^*|  |f^  |  |^-|  |f^-Pkf^ 

(k) 

problem  (3.3)  can  be  restated  as:  find  (f  )N  such  that 

||f<1°-(f00)Nll  -  infTeD  ||-  f<k>-Pkf<k>[j  _  (3.5) 

it  /v) 

with  T  then  provided  by  the  knots  for  (fv  ')N*  In  view  of  Theorem  1 
this  has  the  following  Immediate  consequence. 

Corollary .  Let  f^  and  f o  be  two  regression  functions  satisfying  (3.4) 

with  f^eCk[0,l]  and  f^k^  positive  on  [0,1]  for  i  ■  1,2.  Also  let  8^  T  and 
*2 

6  denote  the  BLUE's  of  8  corresponding  to  when  f ,  in  (3.1) ,  _is  f. 

A1  A2 

and  f2  respectively.  Then.  T  and  6^  T  have  the  same  optimal  designs  for 
all  N  if  and  only  if 

k-1  k+i 

fl(t>  •  +  t*kf 2^^  »  tc [0, 1]  - 

Thus,  not  only  does  the  regression  function  dictate  the  properties  of  the 
optimal  designs  but,  conversely,  the  optimal  designs  also  characterize 
the  regression  function  up  to  a  2k-th  order  polynomial.  The  corollary 
can  be  extended  to  a  wider  class  of  processes  through  use  of  results  in  [8] . 
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4.  Concluding  remarks.  Theorem  1  has  a  straightforward  extension  to  the  case 

k 

of  best  L  [0,1]  approximation,  1  <  p  <  00 ,  from  P  .  Using  the  notation  and 
p  N 

results  in  Theorem  1.1  of  [4]  and  the  techniques  of  Section  2  it  is  possible 


to  show  the  following: 

b  1  lc  1  1  Qf* 

Theorem  2.  Assume  that  either  f  ,g  e  W  *  (0,1)  or  else  f  ,g  eW  ’  ’  (0,1)  ("I Lp [0,1] 

and  | f  ^  | ,  jg^  |  are  monotone  a.e.  with  |f^|°  and  |g^|°  integrable  for 
a  *  (k+p  ^)  \  For  p  ■  <*>  also  assume  that  f,geC[0,l].  Tf ,  in  addition, 

(f  ^g^))  >_  o  (or  -(f  ^g^)  >_  0,  as  appropriate)  a.e. ,  then  f  and  g  have  the 
same  optimal  knots  for  all  N  _if  and  only  if  (1.1)  holds. 

k 

Note  that  when  p-2  Theorem  2  implies  Theorem  1  for  P  .  An  analog  of  Theorem  2 
— k 

for  S„  and  1  <  p  <  «  can  be  obtained  using  results  in  [9]. 

N  — 

More  general  versions  of  Theorems  1  and  2  are  desirable.  In  particular, 

(k)  (k) 

relaxation  of  the  conditions  imposed  on  the  sign  change  behavior  off  and  g 
would  be  of  interest.  Unfortunately,  the  asymptotic  behavior  of  the  optimal 
knots  serves  only  to  distinguish  between  functions~'sJhose  kth  derivatives  are  not 
proportional  in  absolute  value.  Thus,  relaxation  of  these  restrictions  will 
require  the  use  of  Information  provided  by  the  optimal  knots  for  finite  N.  To 
illustrate  the  ideas  involved  consider  the  function  f(x)*x,  xe[0»l]  and  define 
g(x)  to  be  f(x)  on  [0,.5]  and  l-f(x)  on  [.5,1].  Then,  for  k*l  and  p«2,  say,  we 
have  |f'|  -  |g'|  so  that  f  and  g  are  indistinguishable  in  terms  of  the  asymptotic 
behavior  of  their  optimal  knots.  However,  the  optimal  knots  can  still  be  seen  to 
separate  these  two  functions  by  comparing  th?  optimal  knots  for  f  when  N»2 ,  viz 
1/3  and  2/3,  to  the  breakpoints  1/4  and  3/4  for  g. 
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